DOCUMENT RESUME 



ED 383 772 



TM 023 834 



TITLE 

INSTITUTION 

SPONS AGENCY 
PUB DATE 
NOTE 
PUB ITiPE 



Math Assessment Alternatives. 

Northwest Regional Educational Lab., Portland, OR. 
Test Center. 

Departnent of Education, Washington, DC. 

30 Jun 92 

21p. 

Reference Materials - Bibliographies (131) 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MFOl/PCOl Plus Postage. 

Curriculum Development; ""'Educational Assessment; 
Elementary Secondary Education; Mathematics 
Instruction; ^Mathematics Tests; Problem Solving; 
State Programs; "Student Evaluation; *Test 
Construction; "Testing; Testing Programs 
'''Alternative Assessment; "Performance Based 
Evaluat ion 



ABSTRACT 

The articles in this annotated bibliography represent 
holdings of the Test Center in the area of assessment alternatives in 
mathematics. Presence on the list does not imply endorsement of the 
practices by the Test Center; rather, articles are made available to 
stimulate thinking and provide ideas. Some entries present informal 
assessments intended for classroom use. The list includes 56 
documents reiated to elementary and secondary mathematics and a 
variety of alternative approaches to assessment. (SLD) 
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MATH ASSESSMENT ALTERNATIVES 
June 30, 1992 



score OF IMTEREST MOTICE 

The E RIC Facidt y h«s mtgned 
this document lor procecsing 



In our /udgcrrwnt.ihti document 
IS also of mteren to the clearing- 
houses noted to the right. Index- 
<r»g should reflect their special 
points of vie«v. 



The following articles represent Test Center holdings to date in the area of assessment 
alternatives in matheniatics. Presence on the list docs not necessarily imply endorsement 
Articles are included to stimulate thinking and provide ideas. Some of the entries are informal 
assessments, and are intended mainly for the classroom. For more information contact Jud> 
Aner or Ann Davis at 503-275-9500, Northwest Regional Educational Laboratory, 101 S.W. 
Main, Suite 500, Portland, Oregon 972C4. 

Algina. James, and Sue Legg (Eds.). Special Issue: The National Assessment of Educational 
Progress. Located in: .lournai of Ed ucational Measurement. 29, 1992. 

This special issue of JEM discusses the National Assessment of Educational Progress 
(NAEP)-hi story, specification of content and design of assessments for 1992 and beyond 
(math, reading, writing), how students are sampled, and how results are reponed. 
Although some articles are somewhat technical, the general pieces on NAEP's histor>', 
and the design of current assessments will be interesting to the general readership. 

The current plans for math include: 

1. Use of calculators for about 70 percent of the test. 

Estimation skills tasks using an audiotape. 



2. 
3. 

4. 



Yes/No questions to determine the extent to which students understand the same 
informatioi*. when it is presented in different forms. 

Constructed response questions in which students are asked to document their 
solutions by drawing their answers, writing explanations, or providing their 
computations. 

Scoring guides for open-ended questions are uilored to each question. Some examples 
are provided. 



(TC#150.6JEM292) 
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Bock,R.D. The 'TonteiU X Process" Classification of Mathematics Items: Conjoint Scoring, 
1987. Available from: University ofChicago, 5801 S.Ellis Ave. Chicago, IL 60637, 
312-702-1234. 

In this paper, the author describes the content specifications for a grade 8 test in 
mathematics used in a study at UCLA. A "content x process" matrix was used to specify 
test content. Content included numbers, algebra, geometry, measurement, and 
probabilit>'/sutistics. Each of these was crossed with procedural skills (calculating, 
rewriting, constructing, estimating, executing algorithms), knowledge of facts and 
concepts (terms, definitions, concepts, principles), and higher-level thinking (proof, 
reasoning, problem solving, real-world applications). Items that represent some of the 
cells in the matrix are provided as examples. All items are multiple-choice. 

(TC#500.5CONXPR) 

California Stole Department of Education. A Question of Thinking, 1989. Available from: 
California Stote Department of Education, P.O. Box 944272, Sacramento, CA 
94244-2720,916.445-1260. 

This report describes the results of 12th grade student assessment using open-ended math 
problems that was part of the California Assessment Program (CAP). The open-ended 
problems were scored using rubrics developed for each problem. These rubrics are 
described, and "anchor" papers for the six scale values for each rubric are provided. 
Although there is a separate rubric for each problem, they are all intended to reflect the 
following dimensions of problem solving: understanding of mathematics, use of 
mathematical knowledge, and ability to communicate about mathematics. 

(TC#500.3AQL1ESO) 

Campbell, D. Arizona Student Assessment Plan, 1990. Available from: Arizona 

Department of Education, 2102 W. Indian School Rd., Phoenix, AZ 95015, 602-264- 
1774. 

The Arizona Assessment Program has several parts: a short standardized achievement 
test, non-test indicators, and performance assessments in reading, math and writing. The 
performance tests are designed to measure the state s Essential Skills. The math portion 
presents an extended problem solving situation that requires short answers, extended 
answers, and explanations of answers. Each extended exercise has its own specific set of 
scoring procedures that involve assigning a point value if various things are present in the 
response. 

(TC#060.3AR1STA) 

Center for Innovation in Education. Math Their Way, 1990. Available from: Center for 
Innovation in Education, 20665 4th Street, Saratoga, CA 95070. 

Math Their Way is an instructional program designed for grades K-2 liiui emphasizes 
manipulatives. Chapter 3 deals with assessment; the suggested assessment activities tie 
into the instructional program. These are suggested "formal assessments" to be used to 
track student progress two to four times a year. They arc really not intended for daily 
use 'There are IK assessments to evaluate three areas- -prenumber concepts and skills. 



number operations, and place value. All assessments are individual and performance 
based. No technical information is provided. 



(TC#070.3MATTH\I') 

Champagne, Audrey B. Cognitive Research on Thinking in Academic Science and 

Mathematics: Implications for Practice andPoUcy. Located in: Diane Halpern (Ed.) 
Enhancing Thinking Skills in the Sciences and MathemaUcs. 1992. A^toWe from: 
Lawrence Eribaum Associates, Publisher, 365 Broadway, Hillsdale, Nj 07642. 

Although this article is not strictly about assessment, it discusses some topics of 
relevance to assessment. Specifically, it has a very nice section on the relationship 
between the tasks given to students and what they can leam. For example, students can't 
learn as efficiently to integrate knowledge if they are never given tasks that require them 
to do this. This also has relevance to designing "authentic" tasks for performance 
assessments. 

(TC#000.6COGRET) 

Charles, R. Evaluating Progress in Problem Solving, 1989. Located in: CQmtPUPiyator> 14, 
2, pp. 4-6. Also available from: The California Mathematics Council, 1414 S. 
Wallis, Santa Maria, CA 93454, 805-9254)774. 

This article presents a rationale for analyzing student open-ended problem solving in a 
systematic fashion. One sample analytical scoring rubric is presented. The traits are: 
understanding the problem, planning a solution, and getting the answer. The author also 
proposes some other questions to ask as one looks at student problem solving: Did the 
student seem to understand the problem? Were the approaches used to solve the problem 
feasible for finding a solution? Does the answer make sense in temis of the question to 
be answered? 

(TC#500.3EVAPRI) 

Clark, David. The MathemaHcs Curriculum and Teaching Program, 1988. Available from: 
Curriculum Development Centre, Canberra, Australia. Also available from: ERIC 
ED 287 722. 

This document was developed to assist classroom teachers to improve their day-to-day 
assessment of mathematics. Content includes: rationale for assessmem alternatives in 
mathematics, instructions for a two-day inservice program using the materials, 
instructions on how classroom teachers can use the materials without training, and a 
series of exercises, formats and ideas for classroom assessment. 

Assessment ideas include: help with systematically recording information from informal 
observations using checklists and "folios" of student work; setting up opportunities for 
assessment by giving students good tasks to do; assessing problem solving; student self- 
reflection; and communicating results. 

This is written in a very user-friendly manner and contains some good ideas, especially in 
the areas of designing tasks, problem solving and self-reflection. We found some of the 
descriptions of activities a little too sketchy. 

(TC#500.3MCTPMA) 
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C611i& Kevin and Thomas Romberg. Assessment of Mathemaiical Performance: An 
Anafysis of Open-ended Test Items, m9. Available from: National Cotter for 
Research in Mathematical Sciences Education, Wisconsin Centw for Education 
Research, University of Wisconsin, School of Education, 1025 W. Johnson St., 
Madison, Wl 53706, 608-263^200. 

This paper discusses Ac implications of research on cognitive development in math for 
designing assessments. This discussion leads up to some general considerations for 
assessment design and a general summary of current assessment trends. Some sample 
test items are provided to illustrate some of the points. Also some sample performance 
assessment-type items arc shown, but they are not critiqued in light of the previous 
discussion. 

(TC#500.6ASSMAP) 

CoUis, Kevin and Thomas Romberg. CoUis-Romberg Mathematical Problem Solving 
Profites, 1992. Available from: Australian Council for Educational Research 
Limited, Radford House, Frederick Street, Hawthorn, Victoria 3122, Australia. 
Also available from: ASHE, P. O. Box 31576, Richmond, VA 23294. 

This assessment devise has 20 open-ended problems to solve-one problem in each of 
five areas (algebra, chance, it. asurement, number, and space) with four questions per 
problem area. Each question is designed to tap a developmental level of formal 
reasoning For example, the "A" question determines whether the student can use one 
obvious piece of information from the item, while the "D" question determines whether 
the student can use an abstract general principle or hypothesis derived from the 
information in the problem. 

Responses to each question are scored right/wrong. The number of correct responses in 
each subject and developmental level determine what to do instnictionally. Suggestions 
are given for instructional strategics for the various developmental levels. No technical 
information is provided. 

(TC#500.3COLROM) 

Commission on Standards for School Mathematics. Curriculum and Evaluation Standards 
for School MathemaHcs, 1989. Available from: National Counal of Teachers of 
Mathematics, 1906 Association Drive, Reston, VA 22091. 

This book contains standards for curriculum and assessment that attempt to create a 
coherent vision of what it means to be mathematically literate. This book has beer, 
quoted extensively and appears to be the current "standard" for what should be in a math 
curriculum. 

The assessment section covers: three statements of philosophy concerning assessment 
(alignment, multiple sources of information, and appropriate assessment methods and 
uses)- seven sections on assessing various student outcomes (e.g., problem solving, 
communication, reasoning, concepts, procedures, and dispositions); and four actions on 
program evaluation (indicators, resources, instruction, and evaluation team). Each ot the 
seven sections on assessing student outcomes briefly describes what the assessment 
should cover and provides some sample assessment tasks and procedures. 

(TC#500.5CURANE) 
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Dalton, Micbad. Oregon Dimensions of Problem Solving, 1990. Available from: Oregon 
Department of Education, 700 Pringle Parkway S.E^ Salem, OR 97310, 503-378- 
4974. 

The Oregon State Department of Education is cunently ^nsoring a consortium effort to 
develop an analytical trait scoring system for open-ended math problems. Two versions 
of the rubric are included; the second version is a modification of the first, based on a 
pilot test. The team is currently trying out the second version on problems across 
multiple grades. The traits.are: 

1. Conceptual understanding of the problem 

2. Procedural knowledge 



3. Problem solving skills and strategies 

4. Communication 

The current version includes a scoring guide for each trait. The ultimate goal is to 
develop a package that also includes anchor performances, although these are not yet 
included. 



(TC#500.3OREDIO) 

EOL'ALS. Assessment Alternatives in Mathematics, 19S9. Available from: University of 
Califomfa, Lawrence Hall of Science, Berkele> , CA 94720, 415-642-1823. 

This document provides an overview of some possible assessment methods in 
mathematics that cover both process and products. Specific examples are provided for 
writing in mathematics, mathematical investigations, open-ended questions, performance 
assessment, observations, interviews, and student self-assessment. Any of the student 
generated material could be self-selected for a portfolio of work. The document also 
includes a discussion of assessment issues and a list of probing questions teachers can use 
during instruction. 



(TC#500.6ASSALI) 

Eraser, Barrv, John Malone, and Jillian Neale. Assessing and Improving the Psychosocial 
Environment of Mathematics Classrooms. Located in: .lournal fPf RtSgartt» m 
Mflthematics Education . 20, 1989, pp. 191-201. 

This article describes the development of a short form of the My Class Inventory to be 
used in sixth grade math classes to measure the psychosocial characteristics of the 
classroom learning environment, i.e., social interactions. 

(TC#500.3ASS1MP) 
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Glaser, Robert. Expert Knowledge and Processes of Tkinkimg. Located in: Diane Halpern 
rEd.). Enhancing Thinking Skflls in the Sciences and Ma thematics. 1992. AvaiUble 
from: Lawrence Ertbaum Associates, Publisher, 365 Broadway, Hillsdale, NJ 
07642. 

In this article the author describes research on expert performance. Although not directly 
about assessment, expen performance can be used to help understand and define the 
target we have for students, which is the first step toward designing assessment For 
example, expert pcrfonnance can be used to develop criteria for evaluating performance 
tasks. 

The author points out that although expertise is very subject-specific, generalizations can 
be made about its nature across subjects: experts perceive large, meaningful patterns, 
have skillful self-regulatory processes, etc. 

A critical point made by the author is that, "Practice, as it comes about in the usual course 
of training, is not necessarily very efficient. On the basis of our knowledge of the 
specific aspects of competence and expertise, we arc not able to find ways to compress or 
shortcut experience.... This is one goal for performance assessment; we help students 
understand current conceptions of the relevant dimensions of a task so that they don't 
have to rediscover this themselves." 

(TC#050.6EXPXr^P) 

Grobe, R.P., K. Cline, and J. Rybolt. Curriculum Based Assessment For Math: A Summary 
of 1990 Field-Test ResuUs, 1990. Available from: Mt. Diablo Unified School 
District, 1936 CarlotU Dr., Concord, CA 94519. 

The pilot project in Mt. Diablo school district entailed scoring open-ended math 
problems holistically for grades 3, 5 and 8. The holistic scale (0-4) defuies an exemplary 
response as: systematic or elegant, organized recording system, complete and accurate, 
and clear and thorough explanation. One problem along with sample student responses is 
presented for each grade level. A rationale for using open-ended problems is also 
provided. 

(TC*500.3MIDIAC) 

Kansas State Board of Education. Kansas Mathematics Standards and 1991 Kansas 

Statewide Pilot Assessment Results, 1991. Available from: Kansas SUte Board of 
Education, Kansas State Education Building, 120 SE 10th Ave., Topeka, KS 66612. 

This document includes an overview of the 1991 Kansas pilot math assessment and a 
description of results. The pilot included both multiple-choice and open-ended problems. 
The performance assessment portion was scored using a four-trait analytic model 
focusing on problem-solving processes (understanding the question, planning, 
implementing the strategies selected, and verifying the results). Each tra=t is rated on a 
six-point scale. Scoring guides and problems arc included. Detailed instructions and 
sample student work are not included in this document. Some data are reported. 

(TC#500.3KASMAS) 
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Knight, Pam. How I Use Portfolios in Mathematics.l^l. Located in: Educational 

Leadershio. 49, pp. 71-72. Available from: Twin Peaks Middle School, Poway 
Unified School District, 14012 Valley Sprinss Road, POway, CA 92064. 



The author describes her first year experimentation with portfolios in her algebra classes. 
She had her students keep all their work for a period of time and then sort duough it to 
pick entries that would best show their effon and learning in algebra and the activities 
that had been the most meaningful. There isiome help with what she dkl to get started 
and discussion of the positive effects on students. There is some mention of performance 
criteria, but no elaboration. 

(TC#530.3HOWI13S) 

Kulm, G. (Ed.) Assessing Higher Order Thinking in Mathematics, 1990. Available from: 
American Association for the Advancement of Science, 1333 H Street NW, 
Washington, D.C. 20005. 

This book contains a scries of anicles that address various topics in mathematics 
assessment. The anicles address three broad topics: 

1 . The rationale for assessing mathematics problem solving and the need to have 
assessment devices that reflect this emphasis. 

2. Issues that come up when trying to assess higher-order thinking skills in 
mathematics. 

3. General discussions of what to assess and how to assess it. 

There are a few examples of actual assessment techniques. The most relevant articles are 
included on this bibliography as separate entries. 

(TC#500.6ASSHIO) 

Larter, S. Benchmarks: The Development of a New Approach To Student Evaluation, 1991. 
Available from: Toronto Board of Education, 155 College Street, Toronto, Ontario, 
Canada M5T 1P6. 

Benchmarks arc student performance assessment Usks tied to Provincial educational 
goals. Each Benchmark lists the goals to be addressed, the task, and the holistic scale 
used to judge performance. Students are also rated on perseverance, confidence, 
willingness, and prior knowledge, depending on the Benchmark. There are 129 
Benchmarks developed in language and mathematics for grades 3, 6, and 8. 

The percent of students in the sample tested at each score point (e.g., 1-5) arc given for 
comparison purposes, as are other statistics (such as norms), when appropriate. Anchor 
performances (e.g., what a "3" performance looks like) are available either on video or in 
hard copy. 

This report describes the philosophy behind the Benchmarks and how they were 
developed. Some sample Benchmarks (without anchor performances) arc provided in the 
appendices. 

(TC#I00.6BENCHM| 
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Lash, Andrea. An Assessment of Mathematical Problem-Solving SkiUs, 19SS. Available 
from: Far West Laboratory, 730 Harrison SL, San Francisco, CA 94107, 415-565- 
3000. 

This monograph describes a study which examined seventh graders' skill in one aspect of 
mathematical problem solving-problcm analysis. Problem analyus includes identifying 
infomfution necessary to solve a problem, separating relevant from irrelevant 
information, identifying intermediate steps, and representing the information in a 
problem with a table or diagram. 

The monograph describes possible assessuicnt procedures for problem analysis (rating of 
open-ended solutions, purposeful multiple-choice), why they selected the latter 
procedure, and tfie types of problems that elicit problem analysis skills. The complete 
instrument is included. 

(TC#510.3ANASSO) 

Lehman, Michael. Assessing Assessment: Investigating a Mathematics Performance 

Assessment, 1992. Available from: The National Center for Research on Teacher 
Learning, 116 Erickson Hall, Michigan State University, East Lansing, MI 48824- 
1034. 

This monograph, by a high school math teacher, describes his attempt to develop a better 
method of assessing algebra problem solving, concepts and skills than traditional paper 
and pencil tests. The assessment technique involved giving students problems to solve as 
a group, and then having them explain their results in front of a panel of judges. Three 
examples of problems arc provided, as is a brief description of the scoring criteria 
(making sense of the problem, and problem-solving strategies), accuracy of results, 
interpreting results, ability to communicate results, and an explanation of what they did. 
However, these criteria arc not elaborated on, and, although samples of student 
explanations are provided, these are used to describe the understandings the teacher 
reached about his students, not to anchor the perfomiance criteria. 

The author also provides a brief summarj, of the strategies he uses to help students 
develop gicater depth in their understanding of algebraic principals and their 
interrelationships-small group cooperative learning, requiring justifications of 
approaches, etc. 

(TC#530.3ASSASI) 

Lesh. R. Computer-Based Assessment of Higher Order Understandings and Processes in 

Elementary Mathematics, 1990. Located in: G. Kulm (Ed.), ASMSailt Higher Qrd?r 
Thinking in Mathematics . Available from: American Association for the 
aSSS H Street NW, Washington, D.C. 20005. 

This article is as much about how meaningful learning occurs and the nature of the 
structure of knowledge in mathematics, as it is about use of computers in nwth 
instruction and assessment. The basic premise is that computer-based tests should not 
simply be pencil-and-paper tests delivered on-line. They should be pan of an uitegrated 
instruction and assessment system that suppons both learning facts and developing the 
meaningful internal structuring of these facts to form a coherent knowledge system. 

The article discusses three things: ( 1 ) principles underlying a modeling perspective of 
learning and assessment (ideas such as: learning and problem-solving situations are 
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intcipretcd by the lcan>cr by mapping them to internal models; and, several alternative 
'collect* RKxlels may be available to interpret a given atuation): (2) five objectives thai 
should be emi^asized in K-12 math (such as going beyond isolated bits of knowledge to 
construct well-organized systems of knowledge, and think about thinking); and (3) 
specific types of assessment items that can be used to measure these deeper and broader 
understandings (such as conceptual networks and interactive word problems). 

Many sample problems arc provided. 

(TC«500.6COMBAA) 

Lester, F.K. An Assessment Model for Mathematical ProbUm Solving, 1988. Located in: 
Teaching Thinking and Problem Solving. 10, 1988, pp. 4-7. Available from: 
Lawrence Eribaum Associates, Inc., Journal Subscription Department, 365 
Broadway, Hillsdale, NJ 07642. 

This article presents a model for assessing both the problem solving performance of 
students and assessing the task demands of the problem to be solved. The dimensions of 
problem solving (which could be used as a scoring rubric(TC#5(X).3ANASSM)) are: 
understanding/formulating the question in a problem; understanding the conditions and 
variables in the problem; selecting the data needed to solve the problem; formulating 
subgoals and selecting appropriate solution strategies to pursue; implementing the 
solution strategy and attaining subgoals; providing an answer in terms of tlic data in the 
problem; and evaluating the reasonableness of an answer. The article describes these in 
some detail. 

The problem features thai can affect a student's success in solving a problem are: the 
type of problem; the strategies needed lo solve it: the mathematical content/types of 
numbers used; and the sources from which data need to be obtained to solve the problem. 

(TC#500.3ANASSM) 

Uster, F.K., and D. Umbdin-Kroll. Assessing Student Growth in Mathematical ProbUm 
Solving, 1990. Located in: G. kulm ff d j. Assessing Higher.Qrder Thinking m 
Mathematics . Available from: American Association for the Advancement of . 
Science, 1333 H Street NW, Washington, D.C. 20005. 

The authors present a model of factors that influence problem-solving performance, and 
discuss several problem-solving assessment techniques. 

A good assessment program in math should collect information about the following: 
affect (attitudes, preferences, and beliefs), and cognitive processes/ability to get the nghi 
answer (both whether they get the right answer, and the strategies used). The program 
should also systematically define and cover the features of usks (problem type, math 
content, required strategics, etc.) since these affect performance and should be reflected 
in instruction. 

In order to gather infomiation on these three categories of factors, the authors briefly 
review: observations, interviews, student self-reports, and holistic and analytic scoring of 
performances. They recommend against multiple-choice questions. 
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This paper is a general theoretical discussion; no actual tasks, problenis or scoring 
guidelines are provided. 

(TC«S00.6ASSSTG) 

Long, Donna. Mathematics Proficiency Guide^ 1991. Available from: Indiana Department 
of Education, Room 229, Sute House, Indianapolis IN 46204, 317.232-9155. 

Although not strictly about assessnienL, thii document has a nice description of 
mathenutics proficiencies at various grade levels tied to specific instriictional tasks. 
Proficiencies include: problem solving strategies, reasoning, communication, developing 
cognitive stniciures, applying matfi across the curriculum, and various knowledges (e.g. 
decimal places, measurement, and geometry). 

(TCISOOJMATPRG) 

Marshall, S.P. Assessing Knowledge Structures in Mathematics: A Cognitive Science 

Perspective, 1990. Located in: S. Legg & J. Algina (Eds.), Cognitive Assessment of 
Language and Mathematics Outcomes . Available from: Ablex, 355 Chestnut St., 
Norwood, NJ 07648. 

This article discusses the implications of recent advances in cognitive science for 
mathematics assessment The goal in using this research to develop assessment 
techniques is to determine the extent to which students have acquiied specific cognitive 
skills rather than merely whether they can correctly solve particular problems. 

Cognitive theory holds that people solve problems by using three knowledge structures- 
declarative (facts), procedural (algorithms and production mles), and schema (frames that 
relate facts and production rules). To solve a problem, a person must first find the right 
schema, must then correctly implement a set of production rules, and must have stored 
correctly the facts and knowledge required to cany out the necessary algorithms specified 
by the production rules. Errors can occur in any of these three areas. 

Researchers are currently engaged in specifying these knowledge structures in such detail 
that they can develop computer simulations that can, first, solve problems, and second, 
reproduce student errors by leaving out or altering various pans of the necessar>' 
structures. In this way. enors in student responses can be tracked back to the enoncnus 
structure used. The author specifically mentions work in the area of simple arithmetic 
operations, geometry, and word problem;.. 

Additionally, the author discusses two oti.er ways of assessing these things in students - 
reaction time (to assess how automatic a function is); and multiple -choice problems (e.g.. 
"which of the following probletns can be solved in the same way as the one stated 
above?" to get at schema knowledge). Some time is spent with multiple-choice problems 
to explore various types of problems and the technical issues that arise with them. 

It should be pointed out that all these procedures are experimental; none have progressed 
to the point where there is a final product that can be ordered and installed. 

(TC#500.6ASSKNS) 



Judy Ancr. June 1992 
NWREL. 50.V275-9SOO 



10 



Marshall, S.P. The Assessmumt of Schetma Knowledge S<9r Arithmetic Story FrobUms: A 
Cognitive Science FenpecUve, 1990. Located in: G. Kulin (Ed.), Assessinf Higher 
Ordw Thinking in Mathematics. Available fron: American A CTo d a ti o n for the 
Advancement of Science, 1313 H Street NW, Washinston, D.C 20005. 

The Story Problem Solver (SPS) was created to test a thcMy of memoiy architectur' 
called schemata. Under such theories, human memory consists of netwofks of related 
pieces of information. Each netwoik is a schema, a collection of well-connected facts, 
features, algorithms, skills, or strategies. SPS was developed to support instniction in 
story problems in which students were explicitly taugtiit five problem-solving schcmas 
and how to recognize which schema is r^resented a story problem. The 
computerized problems have students pick out the schema or general solution strateg\' 
that fits the story i^oblem, decide which information in the story problem fits into the 
various frames of the schema, identify the steps needed to solve a problem, and decide 
whether the necessary information is given in the problem. 

(TC#500.3ASSOFS) 

Math Learning Center. Recommendations for Assessment, Visual Mathematics^ 1989. (Full 
reference not available.) 

This document is part of a longer monograph. Unfortunately, the longer source was not 
noted in the version we obuincd. The excerpt briefly discusses using writing activities, 
checklists, quizzes, interviews, and self-evaluation to assess mathematics. The 
appendices contain a list of writing activities, journal exercises and starters, a checklist 
covering important dispositions, interview suggestions, aitd self-evaluation activities. 

(TC#500.3RECFOA) 

McTighe, Jay. Maryland School Performance Assessment System, 1991. Available from: 
Maryland State Department of Education, 200 West Baltimore St., Baltimore, MD 
21201, 30I-333-2390. 

This document includes Mar>Iand s: 

1 . Philosophy for developing performance assessments 

2. Statements of targets in reading, language arts and math 

3. General guidelines for developing periormance tasks 

4. Sample perfonnance assessment situations and scoring guides (with samples of 
student work) for grades 3. 5, and 8 

From this package it appears that Maryland's approach is to present a problem situation 
with several different subproblems. Each of the subproblems focuses on a different 
objective: computation, problem solving, reasoning, communication, etc. Not all of the 
objectives arc represented in the various problem situations. From this document it is 
difficult to judge Lhe comparability of scoring guides across grades and problem 
situations (i.e.. the extent to which the guides are tied to individual tasks). However, for 
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the problem situations given, sample student responses were provided lo illustrate the 
score points. 

(TC«500.3MD$CMA) 

Meltzer, LJ. Surveys of Problem-Solving A Educational SkiUs, 1987. Available from: 
Educator's Publishing Service, Inc., 75 Moulton St, Cambridge, MA 02138. 

Although this is an individual test published primarily for diagnosing leaming disabilities 
for students aged 9-14, it has some interesting ideas that could be more generally applied. 
There are two parts to the test--a more-or-iess standard individualized aptitude test, and a 
series of achievement subtests. The math subtest involves a fairly standard test of 
computation. However, the word problem subtest requires the teacher to score each 
problem on choice of correct operations, ability to complete the word problem, efficiency 
of mental computation, self-monitoring, self -correction, attention to operational signs, 
and attention to detail. 

Another interesting part of this test is that after each subtest is administered, the teacher is 
guided through an analysis of the student's strategies in completing the task-efficiency of 
approaching tasks, flexibility in applying strategies, style of approaching tasks, attention 
to the task, and responsiveness during assessment. In the aptitude ponion of the test, the 
teacher also assesses the student's ability to explain his/her own strategies. 

A review in The Reading Teacher . November 1989, concluded that, since tliere is little 
evidence of validity presented by the author, the test should be used informally for 
classroom assessment The reviewer also states: 'The SPES, rather than attempting to 
measure underiying cognitive abilities, instead appear to emphasize underiying strategy 
awareness and use. This orientation appears to reflect the important recent developments 
in educational thinking emphasizing the child as a problem solver who uses intentionally 
selected strategies to improve understanding and leaming." (p. 176) 

(TC#OI0.3SUROFP) 

Mumme,J. Portfolio Assessment in Mathematics, 1990. Available from: California 

Mathematics Project, University of California, 522 University Rd., Santa Barbara, 
CA 93106. 

This booklet describes what mathematical ponfolios are, what might go into such 
portfolios, how items should be selected, the role of student self-reflection, and what 
might be looked for in a ponfolio. Lx)ts of student samples are provided. Criteria for 
evaluating ponfolios include: evidence of mathematical thinking, quality of activities 
and investigation, and variety of approaches and investigations. 

(TC#500.6PORASI) 

Nicholls, J.G., P. Cobb, E. Yackel, T. Wood, and G. Wheatley. Students' Theories About 
Mathematics and Their Mathematical Knowledge: Multiple Dimensions of 
Assessment, 1990. Located in: G. Kulm (Ed.), AMessiny Hiy her.Order Thinking in 
Mathematics . Available from: American Association for the Advancement of 
Science, 1333 H Street NW, Washington, D.C. 20005. 

This paper reports on a series of studies on student attitudes toward mathematics and 
their relationship to mathematical knowledge and understanding. Dimensions of 
attitudes toward math were: (1) how motivated students are to do math; (2) student 
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beliefs about what causes success in math; and (3) student views of the benefits of ^ 
learning math. All items are included. 

(TC#500.3STUTHA) 

Pandy, Tej. Power Items and the Alignment of Curriculum and Assessment^ 1990. Located 
in: n. Kulm (FAX Assessing Hiy her Order Thinkmy in Mathematics. Available 
from: American Association for the Advancement of Science, 1333 H Street Nw, 
Washington, D.C. 20005. 

The author presents a philosophy and approach for thinking about the development of a 
test of mathematics problem solving, and provides some exan^les of multiple-choice and 
shon-answer "power" questions developed by the California Assessment Program. 

The author maintains that typical content by process matrices used to specify the content 
of tests tend to result in tests that measure minuscule pieces of infomiation tfiat are 
fragmented and nonintcgrated. The author prefers to have assessment tasks that are 
broader in focus and cut across several process/content areas, so that in order to get the 
right answer, students must use skills like organizing information, representing problems, 
and using strategies. 

Multiple-choice or short-answer power questions have characteristics that include: they 
assess essential mathematical understandings and interconnectcdness of mathematical 
ideas, rather than isolated facts and knowledge; they are not directly teachable, even 
though teaching for them will result in good instruction; good teachers looking at the 
questions would feel comfortable and agree that such questions are worthwhile teaching 
goals. 

(TC#500.6POWITA) 

Pandy, Tej. A Sampler of MathemaHcs Assessment, 1991, Available from: California 
Department of Education, P.O. Box 944272, Sacramento, CA 94244. 

This sampler describes the types of assessment that the California Assessment Program 
(CAP) is proposing to support cunncular reforms. Illustrated and discussed are open- 
ended problems, enhanced multiple-choice questions, investigations, and portfolios. 
These four types of activities are intended to measure mathematical understandings that 
students develop over a period of several years. 

This monograph includes a definition of "mathematical power"-the ultimate goal of 
mathematics instruction, guidance in the characteristics of assessment tasks that will 
encourage and measure power, a few sample student responses to problems, and help 
with implementation of alternative assessment. 

All performance-based techniques will use a six-point holistic scale. This scale is briefiy 
described. The scale will be tailored for individual tasks. 

(TC#500.3SAMMAA) 
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Psychological Corporation. IntegraUd Assessment System: Mathematical Performance 
Assessment, Available from: Psychological Corporation, 555 Academic 
Court, San Antonio, TX 78204, 800-2284)752. 

This is a scries of 16 performance tasks designed for use in grades 2-8 (two per grade). 
These tasks include such things as figuring out an answer and explaining how the 
solution was reached, doing a series of solutions based on one scenario, generating a 
general principle and applying it to a new situation, etc. Tasks arc cross-referenced to 
skills such as problem solving, communication, estimation, reasoning, and statistics. The 
examination set we received was a little sketchy on how all of this fits together. 

Problem solutions arc scored either holistically (6-point scale) or analytically (4 traits, 4- 
point scales) using the same rubrics across tasks. Scoring can be done cither locally or 
by Psychological Corp. The examination set includes descriptions of the scales but no 
anchor performances. The anchor set also docs not include technical information. Some 
suggestions are given for planning instruction based on results. 

It appears that this information would be of most use in grades 3 and above, although the 
monograph itself docs not specify grades. 

(TC#500.3INTASM) 

Psychological Corporation. GOALS: A Performance-Based Measure of Achievement, \992. 
Available from: Psychological Corporation, 555 Academic Court, San Antonio, TX 
78204, 800-22S-0752. 

GOALS is a series of open-ended questions designed to supplement the SAT-8. There 
are tests for grades 1-1 1 in language arts/reading, mathematics, science, math, and social 
studies. Each test has 10 items. Items are scored using a four-point holistic system (not 
tied to individual tasks) that emphasizes both process and product. 

The preview booklet used for this review did not mention technical information, but did 
include some sample items. 

(TC#510.3GOALS) 

Randall, C, F. Lester, and P. O'Daffer. How to Evaluate Progress in Problem Solving, 1987. 
Available from: National Council of Teachers of Mathematics,- 1906 Association 
Drive, Reston,V A 22091. 

This monograph attempts to assist educators *with the challenge of developing new 
techniques for evaluating the effectiveness of instruction in problem solving by clarifying 
the goals of problem-solving instruction, and illustrating how various evaluation 
techniques can be used in practice. Goals include: select and use problem-solving 
strategies, develop helpful attitudes and beliefs, use related knowledge, monitor and 
evaluate thinking while solving problems, solve problems in cooperative learning 
situations, and And correct answers. 

Evaluation strategies include: informal observation/questioning and recording results 
using anecdotal records or a checklist (two are provided); interviews (a sample interview 
plan is provided); student written or oral self-report of what happening during a problem- 
solving cKperience (a list of stimulus questions is given, as is a checklist of strategics); 
attitude inventories (two arc given); rating scales (three-trait analytic, and focused 
holistic scales are given); and multiple-choice and completion (sample items are given to 
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assess various problem solving abilities; many of these parallel question types mentioned 
by Marshall, above, to assess procedural and schematic knowledge). 

Many sample problems are provided. 

(TC#500.6HOWTOE) 

Riverside PuMishine Company. The Riverside Curriculum Assessment System: 

PeSSrmlTenstExeriise^^ 1991. Available from: Riverside Publishing Company, 
8420 Bryn Mawr Ave., Chicago, IL 60631. 

The Curriculum Assessment System is a customized test development prograni offered 
by Riverside. The Performance Test Exercise part of the System is a recenUy developed 
portion of this program that requires students to generate answers rather than choose 
^swers. 

Tasks center upon a given scenario, and involve from three to ten problems based on this 
scenario. Student responses to each problem are awarded point values depending on the 
presence of certain features in the response. Thus, criteria are tied to specific tasks. All 
scorinc is done locally. No technical information is provided, so there is no way to judge 
whether the list of skills claimed to be required to do each task really is required. 

(TC#060.3RIVCl)A) 

u»inK«ro T A Th^ DomtoiH kHOwledse Strategy for Mothemoticol AssessmeHt^ IWl . 

^^^^^^^^^^^ 

School of Education, 1025 W. Johnson St., Mtdiion, Wl 53706, 608-263.4200. 

This document provides a brief overview of the "Domain Knowledge" strategy used by 
the National Center for Research in Mathematical Sciences Education to assess math 
knowledge of students. This approach is contrasted to the ^yPJ^^IV "^^^^^ f.J .S^^^ 
Behavior Matiix" approach in which content topics are crossed with behavior (usually 
some fomi of Bloom's taxonomy). The author maintains that this approach is outdated; 
the behaviors dimension falls to reflect contemporary notions of how informauon is 
processed and the content dimension Is an inadequate way to describe what is meant by 
''knowing mathematics." 

The "Domain Knowledge" approach involves making a "map "or network of a concept 
domain This reflects a more integrated and coherent picture about knowledge These 
maps can be used to generate tasks, assessment criteria, and formats that get at both 
correctness of responses and the strategies used to amve at the answer. 

(TC#500.6DOMKNS) 

Rombere Thomas, Ann Zarinnia, and Steven Williams. The Influence of Mandated TesHng 
oSM^mitfcf/iifrrucfton^; Grade 8 Teachers' PercepHons^9W9. 
Natural Center for R«iearch In Mathematical Sdence^^^^^ of 
Education, 1025 W. Johnson St., Madison, WI 53706, 608-263.4200. 

This monograph reports on the flrst of a sequence of studies on mandated testing in 
maJhmatS 'This study was a large-scale questionnaire survey to find out from Grade H 
teachers how influentiul mandated testing was on their teaching of mathemat c . 
results of the study showed that neariy 70 percent of the teachers reported that their 
students take a mandated test. Secondly, because teachers know the fonn and character 
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of the tests their students take, most teachers make changes in their teaching to reflect 
this knowledge. Third, the kind of changes teachers make are in contrast to the 
recommendations made by the NCTM standards. Specific examples are given. 

Although this paper does not describe an alternative assessment device, it does provide 
reasons for seeking alternative ways of assessment math. 

(TC#500.6INFMAT) 

Romberg, Thomas, Linda Wilson, and 'Mamphono Khaketla. An Examination of Six 
Standard Mathematics Tests For Grade Eight, 1989. Available from: National 
Center for Research in Mathematical Sciences Education, School of Education, 1025 
W. Johnson St., Madison, Wl 53706, 608-263.4200. 

This study is a follow-up to the survey of teachers described above. The authors 
analyzed the six tests most commonly cited by the eighth grade teachers in that study as 
being used with their students. The authors conclude that the six standardized tests arc 
not appropriate instruments for assessing the content, process, and levels of thinking 
called for in the NCTM Standards. 



(TC#500.6EXASIS) 

Romberg, Thomas. Evaluation: A Coat of Many Colors, 19SS. Avaitable from: National 
Center for Research In Mathematical Sciences Education, University of WIsconsm, 
School of Education, 1025 W. Johnson St., Madison, WI 53706, 608.263-4200. 

This paper describes the impact of assessment information on decision making and 
describes the ways in which assessment must change if it is to have a positive impact on 
such decisions. 



(TC#500.6EVACOM) 

Romberg, Thomas. Assessing Mathematics Competence and Achievement, 1989. Available 
from: National Center for Research in Mathematical Sciences Education, 
University of Wisconsin, School of Education, 1025 W. Johnson St., Madison, WI 
53706,608.2634200. 

This paper describes the author's view of what it means to be literate mathematically. It 
then describes the instructional and assessment implications of this goal. The author 
believes that we need to assess not only mathematical knowledge but also the structure of 
the knowledge. 

(TC#500.5ASSMAC) 

Schoenfeld, A.H. Teaching Mathematical Thinking and Problem Solving, 1989. Located In: 

L.B. Rcsnick & L.E. Kiopfer (Eds), Toward The Thtnkjni Currituium; Current 

ropnttlvf Research . Available from: Association for Supervision and Curriculum 
DevZpmrnM250 N. Pitt St., Alexandria, V A 22314.1453, 703.549.91 10. 

Although this article is more about defining what mathematical problem solving is than 
about assessment, it presents an interesting visual way to represent how students spend 
their time when solving a problem; it also compares such a plot for good problem solvers 
to u plot for an inefficient problem solver. 
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Essentially, the procedure involves tracking the sequence in which students use different 
steps in the problem-solving process (reading the problem, analyzing the problem, 
exploring a solution strategy, planning, implementing a strategy, and verifying the 
results) and the amount of time spent on each. Good problem solvers spend a lot of time 
analyzing and planning, with many self-checks on "how it is going." Poor problem 
solvers tend to fixate on a possible line of attack and pursue it relentlessly even when it is 
clearly not going well. Additionally, there are very few stops to self-check on how it is 
going. 

(TC#500.5STOWTET) 

Silver, Edward and Jeremy Kilpatrick. Testini Mathematiad Problem Solving. Located in: 
R. Charles and E. Silver (Eds.), The Teaching and Assessing of Mathematical 
Problem Solving . 1988. Available from: National Coundl of Teachers of 
Mathematics, Inc., 1906 Association Drive, Reston, VA 22091. 

This paper discusses two topics: how assessment can inform instructional decision 
making and how it communicates what we value. The authors propose that the National 
Assessment of Educational Progress and many other math tests do not provide the type of 
information needed for the improvement of mathematics instruction. The information 
useful for improvement of instruction would be types of enrors kids make, how automatic 
mathematical processes are, and the cognitive structures and abilities associated with 
expertise in the domain being tests. 

(TC#500.6TESMAP) 

Stenmark, Jean. Mathematics Assessment: Myths, Models, Good Questions, and Practical 
Suggestions, 1991. Available from: National Council of Teachers of Mathematics, 
1906 Association Drive, Reston, VA 22091. 

This monograph was designed for teachers in the elementary grades. It is a collection of 
examples of assessment techniques that fo:us on student thinking. Topics include the 
rationale for new ways of assessing mathematics, the necessity of integrating assessment 
and instruction, designing performance assessments (most emphasis is on designing the 
task, although sample holistic and analytical trait scoring systems are shown), what to 
look for during classroom observations and interactions (including questions to ask to get 
at various types of thinking), portfolios (including types of i :ms to mclude and the types 
of infonnation they can demonstrate about students, and criteria for evaluation), student 
self-assessment, and hints to make it work in the classroom. 



(TC#500.3MATASM) 

Surber, J.R. Mapping as a Testing and DiagnosHc Device, 1984. Located in: CD. Holley & 
D.F. Dansereau (Eds.), Spatial Learni ng Strategles-.Technioues. Applications, and 
Related lisuei . Available from: Academic Press, 6277 Sea Harbor Dr., Orlando, 
FL 32887,800.321*5068. 

The book is a general discussion of the advantages of, and procedures for, integrating the 
production of cognitive networks into instruction. The premise is that knowledge of 
facts, rules, algorithms, etc. is only part of what students need to know. They also need 
to know how these facts fit together to form a body of knowledge. Without knowledge of 
the interrelationships, students are not likely to remember the facts or be able to use them 
correctly when they arc remembered. 
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The Surbcr paper discusses a particular type of cognitive networking scheme-mapping- 
and its use in assessment of knowledge structures. The basic procedure consists of taking 
a completed map for the topic to be tested, and deleting portions in various ways. 
Students then complete the map given various types of cues. 

(TC#000.6MAPASA) 

Vermont Mathematics Portfolio Project. Resource Book^ 1991. Available from: Vermont 
Department of Education, 120 SUte Street, Montpelier, VT 05602, 802-828-3I35. 

This document includes sample performance tasks taken from portfolio entries submitted 
by teachers as part of Vermont's 1991 math portfolio pilot projec t, a resource 
bibliography, and a list of suggested readings. The purpose is to provide colleagues with 
tasks that have worked well with students to promote problem solving. This is meant as 
a companion document to the Teacher's Guide CTC#500.3TEAGUI). 

(TC#500.3RESBOO) 

Vermont Mathematics Portfolio Project. Teacher's Guide^ 1991. Available from: Vermont 
Department of Education, 120 State Street, Montpelier, VT 05602, 802-828-3135. 

This document presents Vermont's current view of what should go into a mathematics 
portfolio, provides detailed information about the scoring criteria for portfolio entries and 
the portfolio as a whole, discusses how to develop tasks that will invite student problem 
solving, and provides help with how to manage the portfolios. This is a companion piece 
to the Resource Book (TC#500.6RESBOO). 

(TC#500.3TEAGUI) 

Vermont Mathematics Portfolio Project. Grade Eight Benchmarks, 1991. Available from: 
Vermont Department of Education, 120 State Street, Montpelier, VT 05602, 802- 
828-3135. 

This document provides lots of samples of grade eight student work that illustrate 
different scores for each of the seven analytical traits used in the Vermont Mathematics 
Portfolio Project. Samples were taken from the 1991 ponfolio pilot. 

(TC#S00.3GRAEIB) 

Vermont Mathematics Portfolio Project. Grade Four Benchmarks, 1991. Available from: 
Vermont Department of Education, 120 State Street, Montpelier, VT 05602, 802- 
828-3135. 

This docutticnts provides lots of samples of grade four student work thai illustrate 
different scores for each of the seven analytical traits used in the Ventiont Mathettiatics 
Ponfolio Project. Samples were taken from the 1991 portfolio pilot. 

(TC#500.3(;RAF()B) 
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Vermont Mathematics Portfolio Project. Looking Beyond The Answer'-The Report of 
Vemont's Mathematics Portfolio Assessment Program, 1991. Available from: 
Vermont Department of Education, 120 SUte Street, Montpdier, VT 05602, 802- 
828-3135. 

This repon describes the results of the pilot year of the Vermont's grade 4 and 8 
mathematics portfolio system used for large-scale assessment. The report contains 
information on the rationale for the portfolio approach, a description of what students 
were to include, a description of the criteria used to evaluate the portfolios (with sample 
student performances to illustrate the scoring scale), the scoring and training process, 
results, and what was learned about large-scale assessment using portfolios. 

(TC#500.3REPOFV) 

Webb, N. Alternative Strategies for Measuring L»gher Order Skills in Mathematics: The 
Role of Symbol Systems, 1991. Available from: CRESST, University of California, 
Los Angeles, 145 Moore Hall, Los Angeles, CA 90024. 213-825.4711. 

This document presents an overview of a study that is currently taking place at CRESST 
in which students are asked to represent problems in various equivalent ways (graphs, 
tables, equations, word problems, and diagrams). The premise is that if a student really 
undersunds a problem, he or she should be able to solve the problem presented m any . 
format, and translate from one format to another. Examples arc provided of problems 
represented in different ways. 

(TC#500.6ALTSTF) 

Wells, B. G. Journal Writing in the Mathematics Classroom, 1990. Located in: 

Communicator, 15, 1, pp. 30-31. Available from: California Mathematics Council, 
Ruth Hadlcy, 1414 South Wallis, Santa Maria, CA 93454, 805-925-0774. 

This brief article describes one method that a teacher uses to elicit thinking on the part of 
high school math students. The teacher puts a short phrase on the board at the beginning 
of each class period and students write what they know about that phrase as the teacher 
takes attendance. Sample "prompts" and student responses are included. Although no 
criteria for evaluating responses is included, this article is added here because it 
reprtsents an attempt to tfo writing in math, and because some of the prompts are 
designed to elicit metacognltion, e.g., "What three problems on the final should have 
been eliminated and why?" or "What mathematical fact, concept, skill or msight that you 
learned In this class this year are you most likely to remember and why?" 

(TC#500.6JOUWRI) 

Whetton, C, G. Ruddock, S. Hopkins, et al. Standard Assessment Tasks for Key Stage I, 
1991. Available from: HMSO Publications Centre, P.O. Box 276, London, SW8 
5DT, England. 

In spring 1991, all seven-year-olds in England and Wales (N«600,000) where to have 
been tested using a set of perfonnance assessments tied to a new National Cuiriculutn. 
Areas tested included reading, writing, spelling, handwriUng. math, and science. Ttie 
assessment consisted of a series of usks given to students. For each usk, students were 
assessed on several "statements of attainment (SoA)" (goals In the curriculum). In math, 
thirty-eight SoAs were covered in 19 tasks. SoAs Included those that are falriy 
traditional (e.g., "use addition and subtraction facts up to 10") but also Included some 
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sclf-reflcction and problem solving (e.g. "talk about own work and ask questions," "make 
predictions based on experience." "explore and use the patterns in addition and 
subtraction facts to 10"). 

(TC#070.3STAAST) 
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